Add instrumentation for Azure Service Bus #1225

russcam · 2021-03-16T05:27:05Z

This PR adds instrumentation for Azure Service Bus when an application is using Microsoft.Azure.ServiceBus 3.0.0+ or Azure.Messaging.ServiceBus 7.0.0+ nuget packages.

Two IDiagnosticListener implementations, one for Microsoft.Azure.ServiceBus and another for Azure.Messaging.ServiceBus, create transactions and spans for received and sent messages:

A new transaction is created when

one or more messages are received from a queue or topic subscription.
a message is receive deferred from a queue or topic subscription.

A new span is created when there is a current transaction, and when

one or more messages are sent to a queue or topic.
one or more messages are scheduled to a queue or a topic.

A terrform template file is used to create a resource group, Azure Service Bus namespace resource in the resource group, and set RBAC rules to allow the Service Principal that issues the creation access to the resources. The Service Principal credentials can are sourced from a .credentials.json file in the root of the repository for CI, and from an account authenticated with az for local development. A default location is set within the template, but all variables can be passed using standard Terraform input variable conventions.

Closes #1157

russcam · 2021-03-16T05:33:08Z

An outstanding item is to set the distributed tracing data for receive/receive deferred messages. Messages are typically retrieved and sent in batches however, where each message may represent its own distributed trace. elastic/apm#122 proposes introducing a links property to capture each. Both Microsoft.Azure.ServiceBus and Azure.Messaging.ServiceBus expose the linked traces.

apmmachine · 2021-03-16T06:20:20Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Build Cause: Pull request #1225 updated
Start Time: 2021-03-30T06:23:37.126+0000
Duration: 58 min 22 sec
Commit: 5afeb85

Test stats 🧪

Test	Results
Failed	0
Passed	19155
Skipped	64
Total	19219

Trends 🧪

russcam · 2021-03-25T00:07:00Z

Going to rebase this against master, to pick up #1240 to run tests on CI

Integration tests that run azure resources will be skipped if credentials are not available.

Not needed

gregkalapos

I had a first look, very nice 👍

I added some comments below. On the central config part: I'm not sure it's worth the effort to make the new config runtime changeable right now in this PR - if it turns out to be more work, I think it's totally fine to do it in a follow up - just update the doc in that case.

gregkalapos · 2021-03-29T19:12:24Z

docs/configuration.asciidoc

+
+[float]
+[[config-ignore-message-queues]]
+==== `IgnoreMessageQueues` (added[1.9])


Suggested change

==== `IgnoreMessageQueues` (added[1.9])

==== `IgnoreMessageQueues` (added[1.10])

src/Elastic.Apm.AspNetCore/DiagnosticListener/AspNetCoreDiagnosticListener.cs

src/Elastic.Apm.Azure.ServiceBus/AzureMessagingServiceBusDiagnosticListener.cs

src/Elastic.Apm.Azure.ServiceBus/ServiceBus.cs

src/Elastic.Apm.Azure.ServiceBus/AzureMessagingServiceBusDiagnosticListener.cs

gregkalapos · 2021-03-29T19:47:02Z

src/Elastic.Apm/BackendComm/CentralConfig/CentralConfigFetcher.cs

@@ -266,6 +266,8 @@ internal WrappingConfigSnapshot(IConfigSnapshot wrapped, CentralConfigReader cen

 			public string HostName => _wrapped.HostName;

+			public IReadOnlyList<WildcardMatcher> IgnoreMessageQueues => _wrapped.IgnoreMessageQueues;


Here we'll need something like this:

_centralConfig.IgnoreMessageQueues ?? _wrapped.IgnoreMessageQueues;

_centralConfig.IgnoreMessageQueues doesn't exist yet, that's what we should implement and parse in CentralConfigReader, otherwise it'll only return values from the local config.

src/Elastic.Apm.Azure.ServiceBus/MicrosoftAzureServiceBusDiagnosticListener.cs

github-actions bot added the agent-dotnet label Mar 16, 2021

russcam added enhancement New feature or request v1.9.0 and removed agent-dotnet labels Mar 16, 2021

russcam mentioned this pull request Mar 18, 2021

Use current activity if there is one #1228

Merged

russcam marked this pull request as ready for review March 24, 2021 05:38

russcam requested a review from gregkalapos March 24, 2021 05:38

russcam mentioned this pull request Mar 24, 2021

Add spec for instrumenting Azure services elastic/apm#426

Merged

russcam added 18 commits March 25, 2021 10:12

Instrument send

6d6aba3

Move ConnectionString parsing into environment class

3667cc5

Source azure credentials from file or environment variables

60b340d

Integration tests and IgnoreMessageQueues configuration value

3e74b48

Microsoft.Azure.ServiceBus integration

7c627e0

Rename to Elastic.Apm.Azure.ServiceBus

09e4382

Use PropertyFetcher to get properties

61b9848

Add packaging information

34edd2f

test with minimally supported versions

06feaf8

Add documentation

bf4b7de

add example credential file

5f1d28f

Use DiagnosticListenerBase

1626540

Create and reuse same Service instance for message transactions

aa66bd8

Run terraform locally with authenticated az account

bb29131

Integration tests that run azure resources will be skipped if credentials are not available.

delete example credentials file

d9981b9

Not needed

add table terminator

f003524

Load credentials in CI from .credentials.json

af7d1a2

Add Azure Service Bus sample application

1fb40df

russcam added 2 commits March 25, 2021 10:12

Extract service bus related values out to constants

1cfac4e

Add Azure Service Bus to NetCoreAll

995f266

russcam force-pushed the feature/azure-service-bus branch from b0de537 to 995f266 Compare March 25, 2021 00:13

russcam added 3 commits March 25, 2021 10:19

Fix rebase

608db10

Install terraform for CI

bed17d7

install docker first

a4aadd9

russcam mentioned this pull request Mar 26, 2021

Azure storage integration #1247

Merged

gregkalapos reviewed Mar 29, 2021

View reviewed changes

russcam added 2 commits March 30, 2021 16:22

Address PR feedback

5afeb85

Address PR feedback round 2

52fad70

russcam merged commit 63df486 into elastic:master Apr 6, 2021

russcam added v1.10.0 and removed v1.9.0 labels Apr 6, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add instrumentation for Azure Service Bus #1225

Add instrumentation for Azure Service Bus #1225

russcam commented Mar 16, 2021 •

edited by zube bot

Loading

russcam commented Mar 16, 2021

apmmachine commented Mar 16, 2021 •

edited

Loading

Build stats

Test stats 🧪

Trends 🧪

russcam commented Mar 25, 2021

gregkalapos left a comment

gregkalapos Mar 29, 2021

gregkalapos Mar 29, 2021

	==== `IgnoreMessageQueues` (added[1.9])
	==== `IgnoreMessageQueues` (added[1.10])

		@@ -266,6 +266,8 @@ internal WrappingConfigSnapshot(IConfigSnapshot wrapped, CentralConfigReader cen

		public string HostName => _wrapped.HostName;

		public IReadOnlyList<WildcardMatcher> IgnoreMessageQueues => _wrapped.IgnoreMessageQueues;

Add instrumentation for Azure Service Bus #1225

Add instrumentation for Azure Service Bus #1225

Conversation

russcam commented Mar 16, 2021 • edited by zube bot Loading

russcam commented Mar 16, 2021

apmmachine commented Mar 16, 2021 • edited Loading

💚 Build Succeeded

Build stats

Test stats 🧪

Trends 🧪

russcam commented Mar 25, 2021

gregkalapos left a comment

Choose a reason for hiding this comment

gregkalapos Mar 29, 2021

Choose a reason for hiding this comment

gregkalapos Mar 29, 2021

Choose a reason for hiding this comment

russcam commented Mar 16, 2021 •

edited by zube bot

Loading

apmmachine commented Mar 16, 2021 •

edited

Loading